Summary 


It  is  shown  that  the  functional  equation  technique  of 
the  theory  of  dynamic  programming  may  be  used  to  derive 
functional  differential  eauations  for  the  characteristic 
values  of  integral  -  V  K  ( *i .  t  )n  ( *  )  V*  i 

similar  to  those  obtained  for  the  eigenvalues  of  differ¬ 
ential  equations 
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1.  Introduction . 


In  a  series  of  papers  [1],[2],[  3),  and  in  two  mono¬ 
graphs,  m.ts],  we  have  treated  various  aspects  of  the 
mathematical  theory  of  multi-stage  decision  processes,  a 
rather  imposing  title  which  we  have  shortened  to  'dynamic 
programming' .  In  several  of  these,  notably  [6],  [7], 
we  have  shown  that  the  calculus  of  variations  may  be  viewed 
as  a  multi-st;  ge  decision  process  of  continuous  type,  and 
that  the  functional  equation  technique  which  we  have  employed 
in  connection  with  various  other  continuous  processes  may  be 
utilized  in  this  discipline  to  yield  new  results  and  new  deriv¬ 
ations  of  old  results  in  the  one -dimensional  case. 

In  the  process  of  extending  these  methods  to  include  multi¬ 
dimensional  variational  problems,  the  theory  of  functional  ana¬ 
lysis  enters  in  a  very  natural  way  since  boundary  and  initial 
values  are  now  functions  rather  than  numbers. 

It  turns  oat  that  an  excellent  testing  ground  for  these 
new  tools  i.-  the  theory  of  integral  equations  which  combines 
t lie  new  aspects  of  functional  analysis  with  greater  analytic 
simplicity  than  afforded  by  partial  differential  equations.  In 
addition,  the  results  are  of  independent  interest. 

rfe  shall  sxetch  below  an  annli cation  of  the  theory  of  dynamic 
programming  to  the  problem  of  obtaining  a  functional  equation  for 
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(1.1) 


the  characteristic  values  of  the  integral  equation 

T 

>u(t)  ■  ^  K(a,t )u(s)ds 

o 

Following  this  line  of  thought,  we  shall  in  a  subsequent 
work  develop  corresponding  results  for  some  multidimensional 
problems  in  the  calculus  of  variations. 

2.  Causali  IX  and  Optimality. 

As  is  discussed  and  illustrated  by  means  of  a  wealth  of 
examples  in  Hille's  treatise,  [8],  a  large  part  of  analysis  may 
be  very  properly,  profitably  and  naturally  subsumed  under  the 
general  theory  of  semi-groups  of  operations.  Here  the  basic 
functional  equation  for  a  multi-stage  process  is 

f (P;  s  ♦  t  )  -  f(  P(s);  t),  (2.1) 

which,  as  remarked  by  Hadamard  ,  expresses  the  principle  of 
causality,  cf.  Kille,[9j. 

If  we  consider  a  multi-stage  decision  process,  where  there 
are  many  alternative  routes  to  be  pursued,  the  analogue  of  the 
above  equation  is 

f(P;  s  ♦  t  )  •  Max  f(  P(s,Q);  t)  (i.2) 

Q 

which  expresses  the  principle  of  optimality,  cf  OJ.UJ. 

As  we  have  shown  elsewhere,  [s],  M.  various  results 

in  the  calculus  of  variations  and  in  eigenvalue  problems  connected 
with  differential  equations  may  be  obtained  from  (2.2)  and  its 


limiting  form  as  s-^G,  the  "infinitesxal  generator"  of  (2.2). 

We  now  turn  to  the  application  of  («:.i)  to  the  theory  of 
integral  equations. 


3.  Integral  Equations. 

Let  K(s,t)  be  a  symmetric  kernel  belonging  to  L*  over 
the  square  0<  s,t<_T,  which  is  assumed  to  be  positive  definite. 

The  characteristic  values  of  (1.1)  are  then  equivalent  to  the 
relative  minima  of  ^  u^dt  subject  to  the  constraint  that 
H  K(s,  t )u(s)u(t )dsdt  ■  1.  In  order  to  convert  this  minimization 

O 

problem  into  one  with  the  rroper  invariant  qualities,  we  employ 
the  device  used  in  [6]  and  CO  and  imbed  the  problem  within  the 
larger  problem  of  determining  the  relative  minima  of  the  functional 


i 

J(a;  r( t ) ,u( t ) )  -  \  u^ltjdt, 

subject  to  the  constraint 

T  T  T 

S  ^  K( s ,t )u( s )u( t )dsdt  ♦  d  \  r(t  )u(t)dt 


0.1} 


0.2) 


where  r( t )  €  L^ (0 ,T) .  Let  us  for  the  sake  of  simplicity  consider 
only  the  abs  lute  minimum  and  define  the  function  f(  . ; r ( t ) ) 
follows: 


f(a;  r ( t ) )  ■  Mpx  J v a ; r( t ) , a( t ) ) 


(3.3 ) 


The  function  f  is  "ur.e  for.ctior.  ieo  lignes''  in  t..e  sense  of 
Vol terra  ,  &0. 

Let  us  now  proceed,  using  the  principle  of  optimality  em¬ 
bodied  in  (...£)  to  obtain  ai.  approximate  functional  equation  for 
f,  under  the  assumptions  of  continuity  for  K(s,t)  ^nd  r(t)  as 
functions  of  s  an  i  t. 


3 


Let  h  bt  a  small  positive  quantity#  Then,  setting 
for  tyoographical  convenience  v  ■  u(a),  we  have 

T 

i £ )  f (a ; r( t ) )  -  hvfc  ♦  ^ h  u^dt  ♦  o(h), 


T  T  .T 

(b)  S  S  X(s,t)u(s)u(t  )dsdt  ♦  2\[r(t)  ♦  hK(a ,t )]  u(t  )dt 

cv.W. 


(3.4) 


1  -  hr(a) v  ♦  o(n) , 


Introducing  the  change  of  variable,  u(t)  ■  (l-hr(a)v/2)w( t ) , 
in  order  to  normalize  the  relation  of  (3.4b),  v/e  see  that  the 
problem  of  choosing  u(t)  in  [Th.T]  is  a  problem  of  precisely 
the  same  form  as  the  original*  Hence  we  obtain  the  approximate 
functional  equation 

f('»;ritj)  -  Min^hv***  (l-hr(a )v ) f (a*h ; (r( t )*hK(a ,t ) (l^hr(a ) v/ 
v 

♦  o(h).  (3.5) 

To  obtain  a  limiting  functional  equation  of  differential 
form,  we  must  employ  functional  derivatives. 

4.  A  Gateaux  Lif ferar.tinl  Equation  # 

It  t;  ippropriate  functional  derivative  to  employ  here  is  tl* 
Gateaux  derivative,  rather  than  the  Volterra,  We  shall  employ  es¬ 
sentially  the  rotation  of  HiLle,  [10  J.  and  write  $y, (f)  for  the 
Gateaux  derivative  of  f  with  increment  h, 

S,.(f)  *  lim  f  ( a ;  r  [  t )  ♦  fch(t))  -  f(a;r(t)) 

e 


(4.1) 


Using  this  notation,  the  limiting  form  of  (3*5)  is 

fa  -  -  MJn  [v2  ♦  vr(a)  Srit)(f)  *  VK(fi(l)(n]  ,  U.2) 

where  f  is  the  usual  oartial  derivative., 
a 

This  functional  differential  equation  may  be  used  to  pro¬ 
vide  a  sequence  of  successive  approximations  to  f,  which  can  be 
made  to  converge  montonically  if  we  choose  an  initial  approxim¬ 
ation  in  policy  rather  than  function  space,  cf.  (»J. 


Furthermore,  due  to  the  simple  dependence  of  the  expression 
in  U.«:)  upon  v,  the  function  f  may  be  eliminated  and  an  equation 
for  v  ■  v(a;  r(t ) ) ,  also  involving  Gateaux  differentials,  derived.. 
This  equation  will  have  a  '‘characterise c  theory",  and,  ns  in  the 
Case  of  the  calculus  of  variations,  the  characteristics  will  be 
associated  with  the  Euler  equation  obtained  by  classical  variation¬ 
al  techniques.  These  topics  will  be  discussed  in  subsequent  papers 0 
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